Human/computer interfaces take many forms and serve a wide variety of purposes. A common example of a human/computer interface is the mouse/keyboard/computer screen interface, where a graphical user interface is used for the display of information to a user. Most commonly used human/computer interfaces require a user to explicitly input information to facilitate data entry and decision-making.
Recently, interest has grown in developing human/computer interfaces that take advantage of other means to obtain user input. For example, one system developed by Bradberry, T. J. and Gentili, R. J. and Contreras-Vidal, J. L., in “Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals,” Journal of Neuroscience, 30(9), p. 3432 (2010), which is incorporated by reference as though fully set forth herein, involves an asynchronous brain-machine interface (BMI) for the control of an effector, and uses a linear regression classifier to decode electroencephalogram (EEG) signals to control the two and three-dimensional movement of a cursor. Results show low correlation with the goal of the movement (R^2<0.4) and indicate poor performance, which is common in continuously decoded tasks using EEG signals of brain state.
Another example of a system involving an asynchronous BMI task is in the field of image searching. Non-stimulus driven signals have been used for defining temporal windows for event related potential (ERP) analysis. The system, developed by Cowell et al., uses the onset of a user's eye fixation (as indicated by a predefined length of time where a single location is fixated by the eyes) (see Cowell, A. and Hale, K. and Berka, C. and Fuchs, S. and Baskin, A. and Jones, D. and Davis, G. and Johnson, R. and Fatch, R., “Construction and validation of a neurophysio-technological framework for imagery analysis,” Human-Computer Interaction. Interaction Platforms and Techniques, p 1096-1105, (2007) which is incorporated by reference as though fully set forth herein). This system improves upon the prior art since the definition of a temporal window uses subliminal stimulation times as the onset for ERP analysis. This provides a more accurate and reliable time for onset, whereas fixation time depends on measuring and processing eye position accurately, which is subject to drift, eye blinks, and other artifacts. Moreover, the statistics of fixation time between users are different. Hence, for optimal performance the prior art requires either the training of an additional classifier to learn a user's fixation statistics, or the training of the user to conform to predefined limits when performing the asynchronous BMI task. The selective use of the BMI is not supported by the prior art without artificially constraining the natural pattern of eye movements and fixations of the user. Hence, every fixation stimulus is decoded regardless of its relevance to the task at hand.
Another human/computer interface uses rapid serial visual presentation (RSVP) with a synchronous BMI task. RSVP can be used with a single stimulus presented at a time, such as in object classification, as demonstrated by Gerson et al. (see Gerson, A. D. and Parra, L. C. and Sajda, P., “Cortically coupled computer vision for rapid image search,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), 174-179 (2006), which is incorporated by reference as though fully set forth herein). Another class of application of RSVP uses multiple stimuli, such as in navigation applications, where an environment is navigated by decoding the brain's response to arrows that are flashed. The incorporation of additional stimuli increases the perceptual load on the user, in this case, the complexity of the visual scene during navigation. In addition, the use of multiple stimuli limits the possible commands that can be decoded using RSVP because of increased complexity of the visual scene; and the length of the task increases linearly with the number of stimuli that must be presented for selection.
Other examples of human/computer interfaces, such as those developed by Bell et al. and Furdea et al., are used for the application of spatial goal selection in an image. (see Bell, C. J. and Shenoy, P. and Chalodhorn. R. and Rao, R. P. N., “Control of a humanoid robot by a noninvasive brain-computer interface in humans,” Journal of Neural Engineering, Vol 5, p 214 (2008); and Furdea, A. and Halder, S. and Krusienski, D J and Bross, D. and Nijboer, F. and Birbaumer, N. and Kubler, A., “An auditory oddball (P300) spelling system for brain-computer interfaces,” Psychophysiology, 46(3), 617-625 (2009), both of which are incorporated by reference as though fully set forth herein).
These are examples of selecting a target from a discrete set of locations using RSVP with multiple stimuli. RSVP requires the focused attention by the user on the stimuli presented (otherwise they can be missed) and requires the continuous engagement of brain state for the BMI task, resulting in fatigue.
A need currently exists for a human/computer interface that includes brain monitoring such as EEG signal analysis, and also provides context-relevant decision-related stimuli in a manner that is minimally disruptive to a user's attention (requires a low mental/cognitive load), yet is effective in assisting the user in making useful decisions. The present invention solves this need as described in the sections below.